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Abstract — A denoising technique based on noise invalidation is 
proposed. The adaptive approach derives a noise signature from 
the noise order statistics and utilizes the signature to denoise the 
data. The novelty of this approach is in presenting a general- 
purpose denoising in the sense that it does not need to employ 
any particular assumption on the structure of the noise-free 
signal, such as data smoothness or sparsity of the coefficients. An 
advantage of the method is in denoising the corrupted data in any 
complete basis transformation (orthogonal or non-orthogonal). 
Experimental results show that the proposed method, called Noise 
Invalidation Denoising (NIDe), outperforms existing denoising 
approaches in terms of Mean Square Error (MSE). 

Index Terms — Thresholding, order statistics, confidence re- 
gion. 



I. Introduction 

Data denoising approaches are well known methods with 
presence in diverse applications and have been studied and 
developed in research areas ranging from communications to 
biomedical signal analysis. In denoising techniques, multi- 
resolution representations of the data is generally used. For 
example, wavelet shrinkage is based on rejecting those wavelet 
coefficients that are smaller than a certain value and keeping 
the remaining coefficients. Thus, the problem of removing 
noise from a set of observed data is transformed into find- 
ing a proper threshold for the data coefficients. The pio- 
neer shrinkage methods, such as VisuShrink and SureShrink, 
propose thresholds that are functions of the noise variance 
and the data length [1], [2]. Over the past fifteen years, 
several thresholding approaches such as [3]-[7] have been 
developed. These methods provide optimum thresholds by 
focusing on certain properties of the noise-free signal, and 
they are proposed for particular applications, mostly for the 
purpose of image denoising. Unlike these approaches, the 
method presented in this paper focuses only on the properties 
of the additive noise. By relying on the noise statistics, the 
method defines a probabilistic region of confidence for the 
noise coefficients. Consequently, it validates those observed 
coefficients that are out of the noise confidence region and 
contain noiseless dominant parts. In the presence of additive 
colored noise, the required data preprocessing and/or whiten- 
ing filters may damage the desired noiseless data. Dealing with 
the colored noise without whitening is especially critical if the 
noise correlation is due to the non-orthogonality of the basis. 
The proposed method denoises such data by using only the 
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colored noise statistics in the invalidation step. The principle 
behind the proposed approach is simple yet powerful as it 
takes advantage of the properties for the additive noise and 
demonstrates efficiency in applications for general-purpose 
denoising. 

The paper is organized as follows. In Section the con- 
sidered problem is formulated and revisits the philosophy 
of thresholding. Section |III] derives a noise signature from 
the noise statistics. Section |IV] presents the proposed noise 
invalidation approach. In Section |V] simulation results are 
provided, and conclusions are drawn in Section IVll 

Notations: We use capital letters V, W, and 9 for random 
variables. Samples of these variables are shown with small 
letters v, w, and 9. 

II. Problem Formulation and Motivation 

Noise-free data vector of length N, ~ [yi, - ■ ■ , yw]^ 
is corrupted by an additive white Gaussian random process 
= [wi,--- jWat]^ with zero mean and variance of a^. 
The observed data y^ = [2/I7 • • • iUnV is 

y[n] = y[n] + w[n] (1) 

and can be expressed in terms of a desired orthonormal basis 
such that Q 

0[z]=<s„zj^> (2) 

where Si is an element of orthonormal basis S = 
[si, S2, ■ ■ ■ 7 SAf] and the desired unavailable coefficient is 

=<s„y^> (3) 

Therefore, the following holds 

N N 
yN = Y,e,S,^ yN =Y,e,S, (4) 

1=1 1=1 

and thanks to the orthonormality of the selected basis S, the 
noise coefficients. 



N r 1 
in the available coefficients 



(5) 



(6) 



are also independent identically distributed random variables 
with the same mean and variance of the noise 



EiVi) = 0, var{V{) = ct^ l<i<N 



(7) 



The observed coefficients are soft thresholded by threshold 

sgn{e\i]){\e\i]\~Ts) if \e{i]\>Ts 



Otiierwise 
'inner product of real vectors a and b is denoted by < a, fe >= a^fe. 



(8) 
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The estimate of the noise-free data with this threshold is 



N 

i=i 



(9) 



We provide the optimum threshold by a noise invalidation 
process. 

A. Main Idea of Thresholding 

Traditionally, the main idea in thresholding is to choose 
a value for which all the coefficients with absolute values 
smaller than that value are thrown away. The rejected co- 
efficients are the ones that we cannot fully determine their 
contribution to the noise-free data and therefore, we decide 
to dismiss them. Nevertheless, there is always a chance that 
some samples of the pure noise are above the threshold and 
some noise-free coefficients get buried among the discarded 
coefficients due to the additive noise. The existing thresholding 
methods usually focus on a class of signals, such as images, 
to provide a proper threshold and evaluate the quality of the 
thresholds based on some performance criterion such as the 
mean square error (MSE). Among the thresholding methods, 
the ones with a more general assumption on the noise-free data 
are VisuShrink and SureShrink, which rely on some form of 
piecewise smoothness of the wavelet coefficients. The rest of 
the thresholding methods are application oriented, for example 
the BayesShrink or other image denoising methods that use 
properties such as generalized Gaussian distribution (GGD) for 
the noise-free image itself. Another example is the adaptive 
denoising approach in [4] for neural network applications. 

Here, we present a general-purpose thresholding approach 
that does not need to exploit any particular property of the 
noise-free data itself. Consequently, the thresholding approach 
should be invariant with respect to the order of the data. So if 
the data is reordered and put in an ascending order based on 
its absolute value, the optimum threshold should remain the 
same. In this case, choosing the mth coefficient of the ordered 
data as the threshold is equivalent to throwing away the first 
TO — 1 coefficients. 

For a general-purpose threshold, instead of concentrating on 
the properties of the remaining coefficients after thresholding, 
it is logical to focus on the dismissed coefficients. These 
coefficients are discarded because they are attributed to the 
noise or very noisy coefficients. It is rational to equivalently 
state that these coefficients are discarded since they behave 
similarly to a set of coefficients that can be generated by an 
associated Gaussian distribution of the additive noise. In the 
following section we present one of the signatures of a set that 
is generated by this Gaussian distribution q 

Additive Noise Variance: Thresholding methods, such as 
SureShrink and VisuShrink, rely heavily on the value of the 
additive noise variance cr^. In most practical applications this 
value is not known. The estimate of the standard deviation is 
usually provided by MAD approach where a = MAD/. 6745, 

-Probabilistic approach for finding a significant noise-free component in a 
coefficients is discussed in a data denoising approach in [8]. However, in this 
approach a Laplacian prior for a noise-free data is assumed. 
Preliminary work related to the proposed method is presented in [9] 



and where MAD is the median of absolute value of normalized 
fine scale wavelet coefficients [2]. The method provided in this 
paper also requires the estimate of noise variance, and we use 
the MAD method for this estimation. 

III. Additive Noise Signature 

Consider the additive noise random variable y in (|7| with 
zero mean and finite variance. Define the signature function for 
any value z and v as g{z, v) such that the mean and variance 
of this function over V are finite values: 



E{g{z,V)) = Ge{z) 
var{g{z,V)) = Gvar{z) 



(10) 
(11) 



The signature for samples of a random process of length N, 
~ [Vi, • • • , VnY , with IID members that have the same 
distribution of V is defined as 



1 ^ 



(12) 



It follows that the expected value and variance of the signature 
are 



E{g{z,V'')) = Ge{z) 
mr(5(z,y~)) = ^G.,ar{z) 



(13) 
(14) 



Detailed are shown in Appendix |T] For a large data length, 
while the mean is a finite fixed value, the variance becomes 
smaller. The use of such signatures in invalidation of the 
additive noise is explored with the following example. 

A. Signature Example: Absolute Noise Sorting (ANS) 
Consider a noise signature with the following form 



9{z,Vi) 



1 if \v,\ < z 
\vi\>z 



(15) 



In this case, for the signature of the IID random process V'^ 
in (O we have [10]-[12]: 



E{g{z,V'')) 
varigiz.V'')) 



F{z) 

^F(z)(l-F(z)) 



where F{-) is the cdf of absolute value of the additive noise 



F(z)=:20(-)-l 



(16) 



Details are provided in Appendix |ll] <^ 

For each z, the value of g{z,v^) = m/N where m is the 
number of samples of with absolute values smaller than 
z. Equivalently, when sorting , the mth value is the largest 
Vi that is smaller than z. Therefore, when sorting the ViS, the 
index is Ng{z,v^) = m. Figure [T] illustrates the effect of 
sorting and the role of the small variance in providing a noise 
signature. The figure shows the behavior of 100 samples of 
Gaussian noise with unit variance and length 2048. As the 
top figure shows, with a very high probability, the values of 
this data are bounded between ±3cr. However, if we sort the 
same data based on its absolute value in the middle figure. 
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Fig. 1. Top figure: 100 runs of a zero mean Gaussian distribution with unit 
variance and length 2048. Middle: The same 100 runs of the above figure 
sorted based on their absolute values. Bottom: This is the middle figure with 
its vertical and horizontal axes swapped (m = Ng{z, v^)). 



the values collapse in a much denser area. Such behavior can 
be explained by the ANS signature as follows. The bottom 
figure show the result of swapping the horizontal and vertical 
axes of the middle figure. Here the horizontal axis is z and the 
vertical shows 100 samples of Ng{z,v^) where N = 2048. 
As it is expected, these values are around mean NF{z) with 
variance F(z)(l — F{z)). This will allow us to define proper 
confidence regions, with a high probability p, around the noise 
signature. Due to the noise signature structure, these regions 
are considerably smaller than the corresponding confidence 
regions of the Gaussian distribution of the additive noise itself. 
Therefore, for each z and for a high confidence probability p. 




Fig. 2. Solid line: Mean of the noise g{z,V^). Dashed lines are upper and 
lower bounds with confidence probability 0.999997. 

we can find L]y{z) and Un{z) around the mean value of F{z) 
such that 

Pr{LN{z) < g{z, vn) < UNiz)} = p (17) 

For example. Figure |2] shows the bounds on g{z, V^) for 
confidence probability p=0. 999997 and with a = 2.5. 

B. Confidence Region and Gaussian Estimate 

While it is straightforward to make a table of values of 
the boundaries shown in Figure |2] it is possible to use 



Gaussian estimates for the distributions of g{z,VN) for large 
enough values of N as g{z,V]\f) in (fT2l i is average of N 
independent variables. Using the Central Limit Theorem for 
this distribution, we have0 



Pr{- 



\g{z,VN)-F{z)\ 



<!} 



-X^j^F{z)il-Fiz)) 
This estimates the boundaries in (1% to be 



(19) 



Ln{z) ^ F{z) - A\/ ^F(z)(l - F{z)) (20) 



Un{z) = F{z) + ^F(z)(l - F{z)) (21) 

The choice of A should be such that the probability is close 
to one and at the same time the boundary is not very loose. 
In statistics the three-sigma rule, or empirical rule, states that 
for a normal distribution, almost all values lie within three 
standard deviations of the mear0. For a better quality measure, 
the six sigma approach increases the standard deviation to 4.5 
(equivalently p ~ 0.999997). Consequently, we suggest choos- 
ing A such that 3 < A < 5. Interestingly, our experimental 
observation shows that the threshold associated with A = 4.5 
provides the optimum threshold with respect to MSE in 90% 
of cases. 



IV. Noise invalidation with Absolute Coefficient 
Sorting (ACS) 

The coefficients of our observed data is in form of 6*, = 
Vi + 9i which has the same structure as the noise except its 
mean which is the noiseless coefficient Oi. In this case 

E{g{z, e,)) = Pr{\v, + 9,] < z) = H{z, 9,) (23) 
var{g(z, 6,)) = H{z, - H{z, 9,)) (24) 

where 



H{zA) 



.z + I 



(25) 



Details are provided in Appendix ^ 

Figures [3] and |4] show typical behaviors of H{z^ 9i) for vari- 
ous 9iS, and change of noise variance. Sorting the coefficients 
in this case is analogous to calculation of 



1 " 



^The error function is 

1 r 

erf(x) = — 

VTr Jo 

■^The following is known for A = 3, 4.5 and 5 



dt 



(26) 



(18) 



r/(3/V2) = 0.997300203937, er/(4.5/V2) = 0.999997 (22) 
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Fig. 4. Expected value of g{z, ©) when 9 = 15 and the noise standard 
deviations cr = 1, 2, 4 and 6. 



which according to ( |23] ) and ( l24l i has the following mean and 



1 ^ 

£;g(z,e^) = ^E^(^'^^) (27) 

1 ^ 

t;ar(g(2,e^)) = ^ ^ i?(z, ^0(1 - ^ ^.)) (28) 
1=1 

Since the value of H{z, 9i) in dZSl ) is bounded between zero 
and one, the variance of this value is much less than its mean 
for large values of N. Therefore, a dense area will cover 
the sorted data with a high probability. This area becomes 
distinguished from the area covered by the sorted noise-only 
signal as the value of z grows and as the nonzero coefficients 
become effective. This performance is illustrated in Figure |5] 
which shows the area covered by the sorted noisy data for 
Blocks signal for when SNR=5. The figure also shows the 
behavior of the sorted noise-only data. As it can be seen with 
probability 0.99997 there is no overlap between the sorted 
noise and sorted noisy data after a certain value of z. 

A. Noise Invalidation in Application 

Using the noise sorting signature, it is possible to invalidate 
the noisy coefficients with a high confidence. Figure |6] shows 
the application of the method. The confidence region for the 
noise-only data is available upon knowing or estimating the 
noise variance. As the sorted absolute noisy data leaves the 
noise confidence region, it assures that the coefficients are 
becoming more effective than the noisy part of the data. The 
largest z value for which the departure occurs is the optimum 



Fig. 5. The area between the solid lines is the confidence region of sorted 
absolute values of the noisy data coefficients of Blocks signal (SNR=5) 
with probability 0.999997. The area between the dashed lines is the noise 
confidence region with probability 0.999997. 



threshold (T*) for the noise validation problem 

T* = max^^z < x : Ln{x) ^ g{x, 9^) ^ Un{x) (29) 




Fig. 6. Solid line is the sorted absolute values of the observed data coefficients 
crossing upper bound of the noise confidence region at z = 2.6 when the 
observed data is noisy Blocks signal (SNR=5). The area between the dashed 
lines is the noise confidence region with probabihty 0.999997. 



B. Colored Noise and Thresholding 

Corollary: If the additive noise is colored, the expected 
value of g{z, V'^) and g(z, 8^) will remain the same as these 
expected values for the white noise in ( fTSI l and dZTl i. For the 
variance of the sorted noise we have 



N 



1 

7V2 



N 



cov{g{z,VMz.Vj)) (30) 



»=1 J = l,''7^J 



< -F{z){l^F{z)) + 



1 

where 



N 



Rvv{i ~ i) 



Rvv{0) 



)-F\z)] (31) 



(32) 
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with Ryy{0) = 0-2. 

Proof: In Appendix II VI 

The variance for the sorted noisy data is also provided 
in Appendix |IV] As the variance indicates, the wider is the 
autocorrelation of the noise process with itself, the wider is 
the signature region of the noise and noisy data and therefore, 
as it is expected, it may become more difficult to distinguish 
the data from the noise. 

V. Simulation Results 

We perform our denoising methods on noisy versions of 
six standard signals. Blocks, Mishmash, Bumps, and Quad- 
chirp which are the test signals introduced in [1]. Five level 
decomposition with Haar wavelet is chosen for this exper- 
iment. The confidence probability of the methods for noise 
invalidation region is 0.999997. Figure |7] shows the six signals 
and their coefficients. As this figure confirms, the test signals 
represent a wide range of possible coefficient structures. For 
example Figure [8] shows the coefficient distribution of some 
of these signals, while signals such as Blocks have very 
few nonzero coefficients and many coefficients close to zero, 
signals such as mishmash have more uniformly distributed 
coefficients. Blocks and MishMash signals represent two ex- 
treme structures, while QuadChirp have a combined structure 
of both of these signals. We compare the proposed method 
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Fig. 7. From top to bottom: Blocks, Bumps, HeavySin, Doppler, QuadChirp, 
and MisiiMash. Left figures are the signals and right figures are their 
corresponding wavelet coefficients. 



with Visushrink, Sureshrink which are more general-purpose 
thresholding approaches. On the other hand, Sure-LET and 
BayeshShrink are image denoising methods that are perform- 
ing well with the one dimensional signals. We also consider 
these method for comparison. We compare the performance 
of the methods based on their normalized reconstruction mean 
square error (MSE) which is 
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Fig. 8. Coefficient distribution for Blocks, Doppler, Quadchirp and 
Mishmash. 



where ^ is the resulted denoised data and is the 
noise-free data. Table U provides the MSE of the compared 
methods. As the table shows, NIDe performs better than the 
other approaches in most of the cases. 

The methods are compared for a range of SNRs both for 
white or colored noise. Autocorrelation of the considered 
colored noise is shown in Figure |9] The results of average of 
100 runs are provided in Table U It is important to note that 
the MSEs for all the methods have small standard deviation, 
much smaller than the MSE itself. As the table shows, three 
methods NIDe, BayesShrink and Sure-Let are comparable in 
presence of additive white noise. It is worth mentioning that 
NIDe outperforms the other two methods for the more sparse 
signals 90% of times for even a wider range of SNR than 
the range shown in the table. For the non-sparse ones such 
as Quad-Chirp and Mishmash, however. Sure-let performs 
slightly better than the other two methods with additive white 
noise. For the additive colored noise, NIDe is consistently 
outperforming the other methods. 




-,N\ 



(33) 



Fig. 9. Autocorrelation of the colored additive noise. 

Figures [To] show the denoised versions of Blocks with these 
methods for white and colored additive noise. As the figures 
show and Table I confirms, the denoised data with NIDe and 
BayesShrink in presence of a white noise are comparable while 
the other methods have larger MSE. In presence of colored 
noise, however, NIDe method outperforms the other methods. 

VI. Conclusion 

A denoising approach based on direct invalidation of the co- 
efficients is proposed. The invaUdation process uses a signature 
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TABLE I 

Normalized Reconstruction MSE for the Thresholding Methods. Right Table is for the white additive noise and left table is 



FOR the colored ADDITIVE NOISE WITH AUTOCORRELATION IN FIGURe[91 AVERAGED OVER 100 RUNS 
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Fig. 10. Left figure is for additive white noise with SNR=4 and tight figure is for additive colored noise with SNR=14; (a) Noisy Blocks, (b) VISU, (c) 
SURE, (d) BayesShrink, (e) Sure-Let, (f) NIDE 



of the additive noise in the form of a probabiHstic confidence ties of the additive noise and is such that its standard deviation 
region. The signature is defined based on the statistical proper- is much smaller than its mean. In this work we provided one 
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example of such signature which illustrates itself in simply 
sorting the coefficients. It was shown that such a signature 
represents the noise in a dense area. The density of the area 
depends on the noise variance and the data length. The smaller 
is the noise variance and/or the longer is the data length, the 
denser is the signature area. This will enable us to invalidate 
whether every coefficient is noise dominant or data dominant. 
The theory of the method shows its strength for any type of 
noise-free signal. The variance of the signature decreases as 
the data length grows, providing more distinguish and denser 
area. The method denoises in presence of not only an additive 
white noise, but also an additive colored noise This will also 
enable the user to use the proposed thresholding technique 
with any non-orthogonal basis. Simulation results confirmed 
the advantages of the proposed noise invalidation approaches 
in terms of the reconstruction MSE and illustrates its robust 
performance. While the proposed NIDe approach is a pioneer 
method for a general-purpose thresholding, there seems to be 
a great potential in further analysis, study, and expansion of 
such invalidation method for the purpose of denoising. For 
example, it is worth investigating further functions of different 
noise distributions that can serve as the noise signature. In 
addition, for the cases that the noise-free signal belongs to a 
particular class of signals, a potential extension would be to 
combine the statistical properties of the noise-free signal with 
the statistical structure of the additive noise in order to define 
noise signatures for the purpose of probabiUstic invalidation. 

Appendix I 

Mean and Variance of the Signature for IID noise 
For the expected value of this signature from (fTOl i we have 

1 ^ 

E{g{z,V'')) ^ -Y^E{g{z,V,)) = Ge{z) (34) 
1=1 

For the variance of the signature, since Vi and Vj are inde- 
pendent we have 

E{g{z, V,)g{z, V,)) = E{g{z, V,)E{g{z, V^)) (35) 

Therefore, the cross terms in the desired variance are zero and 
from (fTTl i we have 

1 ^ 1 
var{g{z, V^)) = ^ ^ var{g{z, V,)) = -G,ar{z) (36) 



For the variance 

E{g^[z,V^) = 1 X Pr{\V,\ < z) + x Pr[\V^\ > z) 

= Pr{\V,\ < z) = Fiz) (39) 

From ^ and ^ 

var{g{z, V,) = E{g\z, V,)) - {E{g{z, V))f 

= F{z) - (40) 

On the other hand 

E{g{z,VMz,Vj)) = 1 X Pr{m <zk\V,\<z) + 
X Pr{\V^\ > zor \Vj\ > z) = 
Pr{V, < z)PT{\Vj\< z) ^ F'\z) (41) 

Therefore, we have 

EMz, V,) - F{z)){g{z, Vj) ~ F{z))) = (42) 

which sets the cross terms in the variance of g{z^ V) to zero. 
From ( |40l i, (l42b we have 

1 ^ 

var{g{z,V'')) = Y.var{g{z,V,)) 



-F(z){l-F{z)) (43) 



Appendix III 
Mean and Variance of the Data with Sorting 
Signature 



E{giz,9,))=Pr{\v,+9,\ < z) 



a 



Pr{-z - 9, <v,<z- 9,) = 4>{- 

since (t){—a) = 1 — 0(a), we have 

, ,—z — 9i. , ,z + \ 



'-) (44) 



(45) 



Appendix II 

Mean and Variance of the Sorting Signature for 
IID Noise 

For the mean we have 

E{g{z,Vi)) = lxPr{\V,\<z) + QxPr{\V,\>z) 

= Pr{V, < z) = F{z) (37) 

Therefore, 

1 ^ 

E[g{z, V^)) = - ^ E{g{z, V,)) = F{z) (38) 



which results E{g{z,9i)) to be the defined H{z,9i) in ( |25] ). 
For the variance, due to the structure of g{z, 6i), similar to 
what we had for the noise, we have 



Eig^z,9,)) = E{giz,9,)) = Hiz,9,) 



Therefore 



(46) 



var{g{z,9i)) = E{g'{z,9,)) - {E{g{z,9i))f 

^H{z,9{){l-H{zA)) (47) 

Which results in the variance in 
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Appendix IV the two Xi and Xj random variables are independent and with 

Mean and Variance of the Signature for Colored variances (t^{1 + pij) and (t^(1 - pij). As a result, for the first 



Noise 

In this case the autocorrelation between the zero mean 
Gaussian ViS is denoted by i?^„(m). 

.9(2, Wl) 



term of the covariance 



1 ^ 111 



.9(2, W2) 



E{giz,V,){g{z,V,)) = Pr(|l/,| < z & \Vj\ < z) (56) 
< Pr{\X,\ <V2zk \Xj\ < V2z) (57) 



For the expected value of this function we have 
1 ^ 

E{g{z, F^))) = -Yl ^(9iz, V,)) ^ F[z) 



, , , , (58) 

(48) V 1 + P^'J V 1 " P^J 

Figure [TT] show the area considered for the calculation of this 
probability. 



(49) 



which is similar to that of the IID additive noise. However, 
for the variance, since the following holds 



1 ^ 

vaT{g{z, V^)) = var{- ^ g{z, V,)) 



[11... 1] 



N 

var{g{z,Vi)) cov{g{z,Vi)g{z,V2) 
cov{g{z, Vi)g{z, V2)) var{g{z, V2)) 



cov{g 




covig{z,Vi)g{z,VN)) 



Fig. ■ 1 1 . The desirecj ai'ea for calcul ith in of tl e probabilities in (56) and 



var{g{z,VN))) 



N 



we have 



varigizXl)) = ^Fiz)il ~ F{z)) + 



1 

7V2 



N 



X! cov{g{z,Vi)g{z,Vj)) 
where elements of the second term are 



(50) 



A. Noisy Data 

With similar analogy, for the signal in presence of the noisy 
data, we have 



E{g{z,e''))) 



1 

N 



N 



Y,H{z,e,) 



(59) 



i=i 



cov{giz,Vi)g{z,Vj)) = 
E{g{z, V^){g{z, V^)) - E{g{z, V^))E{g{z, V,)) 



where the second term is simply F'^{z), the first term is 
Pr(|K;| < z & \Vj\ < z). For the first term, the joint 
distribution of Vi and Vj is a Gaussian distribution with zero 
mean and variance 



where H{z, 6i) was defined in dZST l. For the variance of the 
(51) sorted absolute value of the noisy data, similarly ( |46] | holds. 
Therefore, structure of the variance is similar to dSOl ): 



1 ^ 

w(5(z,e^))) = —Y^{H{zA)-H\zA)) 



where 



) = 



, var{ 



1 P^J 
P^j 1 



Rvv{i - j) 



(52) 



(53) 



1 



i=i 

N 



cov{g{z,eMz,Oj)) (60) 
For the covariance in the second term 



i?.„(0) 

with Ryy{{)) ~ (T^. The decomposition of the covariance 
matrix is as follows 



1 1 

1 -1 



1 Ptj 

Prj 1 

1 + Pij 
1 - 



1 1 

1 -1 



Therefore, by the following transformation 





1 


■ 1 1 ■ 






Xj 


"71 


1 -1 







(54) 



(55) 



cov{g{z,&,)g{z,Qj)) = 
E{g{z, QMz, e,)) - E{g{z, e,))E{g{z, 6,)) (61) 

we use i5% to calculate E{g{z,Qi))E{{g{z,Qi)) and have 

Eigiz, e^){giz, = PrQV, + e,\ < z k \Vj + %| < z) 

'^)<X,<V2(z-^il^) (62) 



< Pr{V2{-z- 
k v^(-z- 

y2z e, - 



-) < X,j < V2{z 
V2z 



2 



^)) (63) 



Hi- 



+ Pv ' V^C^ + Pv) - P^J ' v/2(l-p„-) 



= ) (64) 
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